Short-term local predictions of COVID-19 in the United Kingdom using dynamic supervised machine learning algorithms

Background Short-term prediction of COVID-19 epidemics is crucial to decision making. We aimed to develop supervised machine-learning algorithms on multiple digital metrics including symptom search trends, population mobility, and vaccination coverage to predict local-level COVID-19 growth rates in the UK. Methods Using dynamic supervised machine-learning algorithms based on log-linear regression, we explored optimal models for 1-week, 2-week, and 3-week ahead prediction of COVID-19 growth rate at lower tier local authority level over time. Model performance was assessed by calculating mean squared error (MSE) of prospective prediction, and naïve model and fixed-predictors model were used as reference models. We assessed real-time model performance for eight five-weeks-apart checkpoints between 1st March and 14th November 2021. We developed an online application (COVIDPredLTLA) that visualised the real-time predictions for the present week, and the next one and two weeks. Results Here we show that the median MSEs of the optimal models for 1-week, 2-week, and 3-week ahead prediction are 0.12 (IQR: 0.08–0.22), 0.29 (0.19–0.38), and 0.37 (0.25–0.47), respectively. Compared with naïve models, the optimal models maintain increased accuracy (reducing MSE by a range of 21–35%), including May–June 2021 when the delta variant spread across the UK. Compared with the fixed-predictors model, the advantage of dynamic models is observed after several iterations of update. Conclusions With flexible data-driven predictors selection process, our dynamic modelling framework shows promises in predicting short-term changes in COVID-19 cases. The online application (COVIDPredLTLA) could assist decision-making for control measures and planning of healthcare capacity in future epidemic growths.


Supplementary methods
We conducted a complete modelling development and validation process for each of the six outcomes as outlined in the main text. Below we use one-week growth rate prediction as an example to illustrate all the steps of the process.
The process included fitting and predicting that are done at LTLA resolution. We used the log-linear regression to fit the data, and considered three sets of candidate predictors: vaccination (two variables), Google mobility (six variables), and Google symptoms (eight base variables + additional variables).
Step 1. Searching for the optimal lags in predictors using the base model As described in Methods, our starting point was the base model that included LTLA, COVID-19 vaccination (two variables), Google mobility (six variables for six locations) and the eight base variables from Google symptoms. Then we searched for the optimal time lags for each of the datasets by comparing the prediction errors (details are provided below) of the following candidate models:

Supplementary
XV, XM and XS refer to the time lag (in weeks) for vaccination, mobility and symptom search data.
Step 2. Searching for the optimal list of Google symptoms predictors Next we kept the time lags as the optimal lags obtained from Step 1, and tested whether adding additional symptoms from the rest of the Google symptoms (165 symptoms) improved the model by assessing prediction errors (details are provided below).
As described in the Methods, there were eight variables for the search trends of common symptoms for COVID-19 (cough, fever, fatigue, diarrhoea, vomiting, shortness of breath, confusion, and chest pain) in the base model. Then we added one additional symptom from the rest symptoms each time (i.e. the model included 8 base symptoms plus 1 additional symptom) and compared the prediction errors. If none of the models outperformed the original model (here being the base model), then that concluded Step 2. Otherwise, we selected the new best performing model as the new "base model" and repeated this process until completing the assessment for all the symptoms.
Step 3. Additional sensitivity analyses by removing one of the three datasets We kept the optimal lags and symptoms obtained from Step 1 and 2, and tested whether removing any one or two of the three sets of predictors improved the model by assessing prediction errors (details are provided below).

Naï ve models
Supplementary Estimating the retrospective 4-week MSE for model selection As described above, we assessed a model based on retrospective prediction errors. We used the retrospective 4week mean squared errors (MSEs) of prediction for assessment. For the ease of presentation, we defined week 1 as the starting week of our data (i.e. w/c 1-June-2020) and defined week t as the "current" week (the most recent week with complete COVID-19 cases). When predicting the changes in the COVID-19 cases in week t+1, we assumed that the COVID-19 cases were only available until week t (similar to the scenario of real-time prediction). Thus, the 4-week period is related to the length of the prediction timeframe. The table below shows the estimation of the 4-week MSE. Each row below in Table S2 corresponds a model using the "historical data" (i.e. in green) to predict the Y in the week in red. For each model, we calculated the average difference (e.g. D1 for model 1) between (log scaled) and (log scaled) across LTLA. The square value of D1 (i.e. 1 2 ) is the MSE for this model. We calculated the 4-week MSE by averaging 1 2 , 2 2 , 3 2 and 4 2 . The model that had the minimum 4-week MSE was the optimal model, and was used to predict for the target week.
Estimating the prospective 1-week MSE for prediction performance Next, we assessed the prediction performance of the different selected models, by predicting the COVID-19 growth rate for the next target week.

Supplementary Table 3
Estimating the retrospective 4-week average MSE and prospective one-week MSE when predicting for week t, using the prediction timeframe of i.  Prospective prediction performance Sensitivity analysis -revised list of symptoms In the sensitivity analysis, "dysgeusia", "anosmia", "headache", "nasal congestion" and "sore throat" were added to the base model to test whether including these symptoms improved the predictive accuracy.
The prospective 1-week MSE of the sensitivity analyses and those of the main analysis for week 2/23 are shown below. Prospective MSEs by LTLAs using the optimal models Supplementary Figure 4 Prospective 1-week MSEs by prediction timeframe, week 1/40. MSE: mean squared error. Panels: 1-week growth rate (A), 2-week growth rate (B) and 3-week growth rate (C) by publication date; 1-week growth rate (D), 2-week growth rate (E) and 3-week growth rate (F) by the collection date of specimen.